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Abstract 

An analytical solution for the time 
optimal control function and optimal trajec- 
tories of a second-order linear system with a 
constant time delay has been obtained. The 
system is a simple spring mass system with a 
constant time delay in the position variable. 
In addition to expressions for the control 
function and trajectories, an equation for 
the limit cycle has been established as well 
as a numerical construction of a pseudo 
switching curve for this system. The optimal 
control function was deduced from the maximum 
principle of Pontryagin for systems with 
delay. The two-point boundary value problem 
associated with the differential -difference 
equation in the synthesis of the control was 
solved by use of a Newton-Raphson iteration 
scheme. The results are expressed in a. form 
which can be easily compared to the well- 
known results of the corresponding system 
without delay. 


Introduction 

Recently, much interest and effort have 
been spent in the attempt to answer questions 
on the stability and control of linear 
systems of the hereditary type; that is, 
systems which are described by linear 
differential -difference equations whose 
future behavior depends upon its past and 
present states. Although these systems were 
studied by Euler in 1750, no appreciable 
application of them to physical situations 
were discovered and employed before the first 
quarter of the 20th century. Physical 
systems which contain delays occur in such 
fields of science as biochemistry, economics, 
traffic flow, control theory, and so forth. 


For example, in the remote control of distance 
space vehicles, the communication delay can 
adversely affect the stability of the system. 
Time delays in engine response of large jet 
transports can seriously affect the handling 
qualities of the aircraft. 

Many questions concerning the stability 
of these systems have been answered by 
Bellman (1955) , Pontryagin (1962), Bellman 
and Cooke (1963), Krasovski! (1963), 

Kashiwagi and Flugge-Lotz (1967), Halanay 
(1966), and Kashiwagi and Shaughnessy (1967), 
just to name a few. Questions concerning the 
optimal control and the controllability of 
systems containing delays have been partially 
answered as well. Khratishveli (1961) has 
extended Pontryagin' s maximum principle to 
systems described by differential -difference 
equations. Balakirev (1962) has applied the 
extended maximum principle to a particular 
linear system with delay to determine the 
switching line. Chyung and Lee (1965) have 
considered the time optimal problem for the 
case of a general controller restraint set. 
Oguztoreli (1966) has considered the control 
of delay systems in general, and in particu- 
lar, has extended the Neustadt method for 
control synthesis to the system with delay. 
Many questions concerning the time optimal 
control of linear systems with delay remain 
to be answered. The purpose of this paper is 
to examine in some detail the basic character- 
istics of a particular linear system con- 
taining a delay - namely, a second -order sys- 
tem free of friction, but which contains a 
time delay in its position feedback. Because 
of the well -developed theory and application 
of this problem without delay, many important 
analogies and differences are discussed. 
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Description of the System and 
the Solution Trajectories 

The mathematical model considered here 
has the form 


( 1 ) 


where A, B, and C are constant matrices, 
and u(t) is the control function which is 
bounded by the inequality j u(t) | ^ 1, 

The solution of equations (l) for the 
state trajectory in terms of the control 
function is given as 


x(t) = Ax(t) + Bx(t 
x(t) = G(t) 


( -9 $ t % C 


x(t) = K(t)G(0) + f° K(t - t - 8)BG(r)d T 

J -0 


+ 



T)Cu(r)dr 


( 2 ) 


where K(t) is the fundamental matrix for the 
differential -difference equation given by 
equations (l) . The fundamental matrix K(t) 
satisfies the following differential- 
difference equation and properties: 


K(t) = AK(t) + BK(t 


0) 


K(t) = 0, t < 0 
K(0) = I 


(5) 


In this paper, one of the simplest examples is 
considered; that is, the case for 


A = 


0 


0 



G(t) 


2 

at + bt + c 
2at + b 



w 


The initial function G(t) was assumed to be 
quadratic in time. This choice makes it pos- 
sible to represent many functions through the 
proper choice of the constants a, b, and c. 


For this system, the elements of the 
fundamental matrix K(t) are 


<-u n 

n=0 


2n 


M(t ) _ _ n Q\2n+l 

*12^ = -k 21 (t + 0) = (-1) (2n + l) ; 


21 v 


(5) 

where H(t) = jj^j meaning the largest integer 
less than or equal to The fundamental 

matrix K(t) for the system behaves somewhat 
like the fundamental matrix $(t) for the 
system without delay. It can be seen from 
figures 1 and 2 that for small values of the 
delay (in this case 0 < O.l), K(t) and $(t) 
are very nearly the same. However, for large 
delays (0 > O.l) striking differences occur 
between K(t) and $(t) . The most obvious of 
the differences is that the magnitudes of the 
elements of K(t) increases rapidly with time. 


If the results of equations (k) and (5) 
are substituted into equation (2), the fol- 
lowing elements of the state vector x(t) 
result : 


x 1 (t) = (a9 2 - b0 + c + 2a)k l;L (t + 0) 

- 2ak xl (t + 20) + (b - 2a9)k 12 (t + 0) 
+ I x £u(t) ,tj 

x 2 (t) = -(a0 2 - b0 + c + 2a)k^ 2 (t) 

+ 2ak 12 (t + 0) + (b - 2a0)k i:L (t + 9) 




All that is needed to reduce the expressions 
given by equations (6) to a completely alge- 
braic form is the integration of the integrals 
1^ and Ig. These expressions can be inte- 
grated immediately, once the optimal control 
function u(t) is determined. 
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The Time Optimal Control Function 
and Its Synthesis 

The time optimal control function for the 
system given hy equations (l) can he obtained 
from the extended maximum principle of 
Pontryagin for systems with delays. According 
to this principle, the optimal value of u(t) 
is that value which maximizes the Hamiltonian 


where i|r 2 (t) can be obtained from the solu- 
tion of equations (ll) and (12). The solution 
for jr(t) is 

±(t) = K'(T - t)£(T) (0 g t % T) (l4) 

The resemblence of this result to the value of 
the adjoint vector for systems with no delay 
is clear. 


H ^(t),x(t),x(t - 9),u(t)J t^tjx^t) 

1=1 ( 8 ) 

where the elements of the adjoint vector >|r(t) 
are determined from the differential -difference 
equations for 0 | t < T - 8 as 


^(■t) = - Qi(t),x(t),x(t - 9),u(tj] 


§ - x ^ t ) [i(t + e),x(t + e),x(t),u(t + e]j 


(i = 1,2) 


(9) 


and for T - 6 ^ t < T 


^(■t) = - g Xj ( t - y [i(t),x(t),x(t - e),u(tj 

(i = 1,2) (10) 


For the system given by equations (l) and (4), 
these equations can be rewritten as 

t ± (t) = ~A'i(t) - B'i(t + 9) 

(0 < t < T - 9) (11) 

i(t) = -A'i(t) 


Substitution of \|< 2 (t) as determined 
from equation (l4) into the above expression 
for the optimal control gives 

u(t) = sgn^ 2 (Tf] sgn[k 11 (T - t) + ak 12 (T - t]] 

(15) 

where a = t 1 (T)/4 2 (T). This form for u(t) 

shows that both components of j/(T) need 
not be known; instead, all that is needed is 

the ratio a and the value of sgnj^ 2 (T)^j. 

The fact that the optimal control function is 
of the bang-bang type allows an evaluation 
of the integral functions and I as 


x l[u(t),t] = sgnp 2 (0+)] 


(-i) m - kll (t + e) 


+ 2 


p=l 


(-l) P+1 k la (t - t p + 9) 


, ju(t),tj 


= sgn 


+ 2 


* 2 (o+) 


k 12 (t) 


(' 1 ) Pk l 2 ( t " ^ 


P=1 


(16) 


(T - 9 g t < T) (12) 

where A' and B' are the transposes of the 
coefficient matrices given in equations (4), 
and T is the optimal time. The value of 
u(t) which makes the Hamiltonian as given by 
equation (8) maximum subject to the constant 

|u(t) | S 1 is 


u(t) = sgn[jf 2 (tJ 


(15) 


The value of m occurring in the above 
expression is the number of switches required 
to control the system, and t is the pth 

switching time as determined from the pth root 
of the equation 


kll (T - t) + ok 12 (T - t) 


where 0 < t^ < t 2 <. 


< t < t ,, <• . . 

P P+1 

It can be seen from equation (17) that the 
value T - t is a function of a only and 
not of T. p 


(17) 

. < T. 
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The values of T and a which serve to 
drive the perturbed state to the origin in 
time T by use of a bang -bang control func- 
tion can be obtained as the unique solution 
for T and a of the set of equations 

x (T,a) = 0 \ 

> (18) 

Xg(T,a) = 0 J 

Since the partial derivatives of and 

Xg(T,a) with respect to T and a can be 

computed analytically, the Newton-Raphson 
scheme of solving for the root of a system of 
equations can be used in equations (18), once 
approximate values for T and a are given. 
Optimal trajectories for several values of 
delay are shown in figure 3- The initial 
function for this example was taken as a 
linear function of time (eqs. (4)) with 
b = c = 2.0. In addition to the obvious dif- 
ferences in trajectory characteristics, there 
is also an increase in the value of the opti- 
mal time required to control the system with 
increasing values of the delay. 


given by a particular root of the equation 
k 1P (T) = 0. For the family of initial func- 
tions given in equations (4) , the value of 
xp(0) and xgCO) are c and b, respec- 
tively. This means that if T r , satisfying 
k 1P (T r ) = 0, is substituted into equations (6) 
with xq(T r ) = Xg(T r ) = 0, the resulting 
values of c and b are the coordinates of 
the rth cusp point. These coordinates 
denoted by x-^ r and Xg r , respectively, 


x l>r - 1 + 


- 2a 


9k (T + 9) 

1 + 5-T3£ +~T 


k (T + 9) k (T + 29) 
* xl 2 (T r + 9) “ k ll^r + 


+ k^f-Tey 


r-1 


(-D 1 


> ( 19 ) 


+ 2^ (-l) P k 1;l (T r -T + e) 
P=1 


Optimal Switching Curve and 
Limit Cycle 

An optimal switching curve can be con- 
structed numerically for the system with delay. 
In the analysis, the term optimal switching 
curve will denote the curve that separates 
the parts of the trajectory for which u = +1 
from those for which u = -1 for a particular 
family of initial functions. This curve 
reduces to the ordinary optimal switching 
curve in the case of vanishing small delay 
values . 


k 12 (T r + 6 5 
k ll< T r + 9 M 

J 

where is the rth root of the equation 

k,„(T) = 0, and 0 < T. < T 0 <. . .< T < 

12 12 r 

T r+1 <. . . . The analytical determination 

of the coordinates of these points is sig- 
nificant to the construction of the 
switching curve since much work is required 
to locate them numerically. 



There are certain points on the switching 
curve which will be referred to as cusp points. 
For the system with no delay these points 
correspond to the intersection of the 
switching curve with the xp axis. Initial 
states with these points as coordinates can 
be driven to the origin in the time required 
for a particular number of consecutive 
switches of the control function. The value 
of a corresponding to these initial states 
are a = +<» or -°° or equivalently 
^ (T) = 0. The corresponding value of T is 
a particular root of the equation 0 p 2 ( T ) = 0. 
In the case of systems with delay, the cusp 
points do not necessarily occur on the Xj 
axis and the value of T for the synthesis is 


An example of the switching curve associ- 
ated with a system whose initial disturbance 
function is linear in time (a=0, b=2.0 

in G(t) is given for 9=0 and 0.3 in fig- 
ure 4. Closely associated with the switching 
curve is the limit cycle for the system. The 
curve is symmetric with respect to the origin 
and its equation for x^(t) <0 is 


x-^t) = (3 - l)k- L1 (t + 9) - 1 

Xg(t) = -(3 - l) k 12 (t) 

where 3 is the limit of the sequence 
whose rth element is 


( 20 ) 




3 r = 1 + k lx (T r + 9) 


(-D 1 


r-1 

sz 

P=1 


+ 2 HI (-l)\ 1 (T r - T p + 9) 


( 21 ) 


The limit cycle is a closed curve in state 
space which separates the space into two 
regions. Disturbed states interior to this 
curve can be controlled with the class of 
time optimal control functions used here. 
Disturbed states exterior to the curve cannot 
be controlled with these control functions. 
Disturbed states which lie on the curve remain 
on the curve indefinitely. An example of how 
the limit cycle of the system described by 
equations (20) varies with increasing values 
of the delay is shown in figure 5 for the case 
of a = 0. As shorn, the effect of increasing 
the magnitude of the delay is to substantially 
reduce the region of state space which can be 
controlled optimally. The extreme cases for 
the limit cycles are a circle whose radius 
becomes infinite as 0 approaches zero (the 
entire state space controllable) and the x^ 
axis between ±1 as 9 approaches infinity. 


Comments on an Approximation 


An indirect and simple method of com- 
paring the results of this approximation to 
the exact analysis just discussed is through 
the fundamental matrices of the two systems. 
Shown in figure 6 are the elements of the 
matrix, plotted versus time for exact calcu- 
lations from equations (l) and approximate 
calculations from equations (23) and (24) . 

The figure indicates relatively good agree- 
ment between the elements of the two matrices 
for small delays (9 < 0.1), but sizable dif- 
ferences for larger values of the delay. This 
implies that the approximation should give 
reasonable results for systems with small 
delays compared to the control time. 


Concluding Remarks 

A time optimal control study of a par- 
ticular linear system containing a delay in 
its feedback position has been conducted. It 
was found that the optimal control function 
is the bang-bang type whether or not a delay 
is present; however, the times at which the 
control function switches sign and the optimal 
time for control are dependent upon the size 
of the delay present in the system. For the 
cases considered here, the control time 
increased monotonically with increasing values 
of the delay. 


An approximation that has been used in 
the past to study systems of the hereditary 
type with small delays is 

x(t - 0) = x(t) - 0x(t) (22) 

which represents the first two terms of a 
Taylor's series expansion of x(t - 0). This 
approximation reduces the hereditary system 
given by equations (l) to an ordinary linear 
system containing negative friction which is 
described by the equations 

x(t) = A Q x(t) + ^(t) 

x(0) = 3^ 

where 



— 



— — . 



— — 


0 

1 


0 



c 




, B = 



and x = 



-1 

9 

0 

1 


“TO 

b 


_ 



— ^ 



_ — 


(24) 


It was found that the switching curve for 
the system with sizable values of delay dif- 
fered considerably from the switching curve of 
the corresponding system without delay. In 
fact, the switching curve for the system with 
delay attenuates along the xq axis . For 
large values of delay it was found that a limit 
cycle exists which substantially reduces the 
region of state space which can be controlled 
optimally. 

The analysis further indicated that the 
approximation of the term containing the 
delay by the first few terms of a Taylor's 
series comparison can be useful in eases for 
which the delay is small compared to the con- 
trol time, but should be abandoned for large 
values of the delay. 
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Figure 1. - Variation of the element k-,-, (t) of the fundamental matrix with time. 




Figure 3.- Effect of time delay on the optimal trajectory in state space. 
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Figure 6.- Elements of the fundamental matrix obtained from exact and approximate analyses. 




